Pictonaut: movie cartoonization using 3D human pose estimation and GANs

This article describes Pictonaut, a novel method to automatically synthetise animated shots from motion picture footage. Its results are editable (backgrounds, characters, lighting, etc.) with conventional 3D software, and they have the finish of professional 2D animation. Rather than addressing the challenge solely as an image translation problem, a hybrid approach combining multi-person 3D human pose estimation and GANs is taken. Sub-sampled video frames are processed with OpenPose and SMPLify-X to obtain the 3D parameters of the pose (body, hands and face expression) of all depicted characters. The captured parameters are retargeted into manually selected 3D models, cel shaded to mimic the style of a 2D cartoon. The results of sub-sampled frames are interpolated to generate a complete and smooth motion for all the characters. The background is cartoonized with a GAN. Qualitative evaluation shows that the approach is feasible, and a small dataset of synthetised shots obtained from real movie scenes is provided.This work is partially supported by the Spanish Ministry of Science and Innovation under contract PID2019-107255GB, and by the SGR programme 2017-SGR-1414 of the Catalan Government.Peer ReviewedPostprint (published version

Assessing the participatory design of a project-based course on computer network applications

New teaching methodologies which foster student involvement, such as project-based learning, are nowadays part of the study curriculum of many engineering schools. Project-based learning courses, however, often build upon other previously taught technical courses, where the technical content for the project to be developed is studied. That type of course design focuses on building the transversal capabilities of students, and the technical challenges of the project are the mean to acquire these non-technical skills. In this paper, we present and assess a project-based course on computer network applications of a computer science school, which has been designed to improve within the same course both the transversal and technical skills of the students. The proposition of interest is that the course not only aims to train the students’ transversal skills by a group work project, but also to practise new technical topics and technologies. We argue that the key element of the proposed course design is that each student project group defines with the instructor the project they would like to develop in the course. We present first the design of the course and then an assessment with questionnaires, which were conducted over two semesters with the students enrolled in the course. The obtained results indicate that the students achieved both technical and transversal skills, while the instructors need to be flexible to adapt to diverse technical topics of the proposed projects.Peer ReviewedPostprint (published version

Multimedia big data computing for in-depth event analysis

While the most part of ”big data” systems target text-based analytics, multimedia data, which makes up about 2/3 of internet traffic, provide unprecedented opportunities for understanding and responding to real world situations and challenges. Multimedia Big Data Computing is the new topic that focus on all aspects of distributed computing systems that enable massive scale image and video analytics. During the course of this paper we describe BPEM (Big Picture Event Monitor), a Multimedia Big Data Computing framework that operates over streams of digital photos generated by online communities, and enables monitoring the relationship between real world events and social media user reaction in real-time. As a case example, the paper examines publicly available social media data that relate to the Mobile World Congress 2014 that has been harvested and analyzed using the described system.Peer ReviewedPostprint (author's final draft

Distributed training of deep neural networks with spark: The MareNostrum experience

Deployment of a distributed deep learning technology stack on a large parallel system is a very complex process, involving the integration and configuration of several layers of both, general-purpose and custom software. The details of such kind of deployments are rarely described in the literature. This paper presents the experiences observed during the deployment of a technology stack to enable deep learning workloads on MareNostrum, a petascale supercomputer. The components of a layered architecture, based on the usage of Apache Spark, are described and the performance and scalability of the resulting system is evaluated. This is followed by a discussion about the impact of different configurations including parallelism, storage and networking alternatives, and other aspects related to the execution of deep learning workloads on a traditional HPC setup. The derived conclusions should be useful to guide similarly complex deployments in the future.Peer ReviewedPostprint (author's final draft

Reliable scholarly objects search and interchange framework

Authors of scholarly objects might fear that there is a potential risk that the original material they publish in online sites or that they submit for evaluation to scientific journals or conferences is used by others as their own material. In such cases, it would not be easy for the original authors to prove authorship of the original contribution. In similar circumstances, it is very difficult to prove the authorship or origin of some materials that are being distributed amongst social networks, private or institutional websites or any other means through the Internet, namely documents, papers, images, data, etc. Those materials can be easily plagiarised (e.g. partially or totally translated) and redistributed without any control and with no means to prove authorship. In this context, we propose an online framework for the registration, search, interchange and trade of scholarly objects, which helps to overcome the potential drawbacks of online distribution and publishing. This framework acts as an intellectual property repository and sales point, where people is able to register content and determine the way they want to trade it, while providing innovative search capabilities based on the MPEG Query Format standard [1]. Creative Commons (CC) [2] limitations are identified and overcome by means of a licensing approach that combines Rights Expression Languages and the MPEG-21 Media Value Chain Ontology [3].Peer ReviewedPostprint (published version

Reliable scholarly objects search and interchange framework

Authors of scholarly objects might fear that there is a potential risk that the original material they publish in online sites or that they submit for evaluation to scientific journals or conferences is used by others as their own material. In such cases, it would not be easy for the original authors to prove authorship of the original contribution. In similar circumstances, it is very difficult to prove the authorship or origin of some materials that are being distributed amongst social networks, private or institutional websites or any other means through the Internet, namely documents, papers, images, data, etc. Those materials can be easily plagiarised (e.g. partially or totally translated) and redistributed without any control and with no means to prove authorship. In this context, we propose an online framework for the registration, search, interchange and trade of scholarly objects, which helps to overcome the potential drawbacks of online distribution and publishing. This framework acts as an intellectual property repository and sales point, where people is able to register content and determine the way they want to trade it, while providing innovative search capabilities based on the MPEG Query Format standard [1]. Creative Commons (CC) [2] limitations are identified and overcome by means of a licensing approach that combines Rights Expression Languages and the MPEG-21 Media Value Chain Ontology [3].Peer ReviewedPostprint (published version

How MPEG Query Format enables advanced multimedia functionalities

In December 2008, ISO/IEC SC29WG11 (more commonly known as MPEG) published the ISO/IEC 15938-12 standard, i.e. the MPEG Query Format (MPQF), providing a uniform search&retrieval interface for multimedia repositories. While the MPQF’s coverage of basic retrieval functionalities is unequivocal, it’s suitability for advanced retrieval tasks is still under discussion. This paper analyzes how MPQF addresses four of the most relevant approaches for advanced multimedia retrieval: Query By Example (QBE), Retrieval trough Semantic Indexing, Interactive Retrieval, and Personalized and Adaptive Retrieval. The paper analyzes the contribution of MPQF in narrowing the semantic gap, and the flexibility of the standard. The paper proposes several language extensions to solve the different identified limitations. These extensions are intended to contribute to the forthcoming standardization process of the envisaged MPQF’s version 2.Postprint (author’s final draft

Towards the cloudification of the social networks analytics

In the last years, with the increase of the available data from social networks and the rise of big data technologies, social data has emerged as one of the most profitable market for companies to increase their benefits. Besides, social computation scientists see such data as a vast ocean of information to study modern human societies. Nowadays, enterprises and researchers are developing their own mining tools in house, or they are outsourcing their social media mining needs to specialised companies with its consequent economical cost. In this paper, we present the first cloud computing service to facilitate the deployment of social media analytics applications to allow data practitioners to use social mining tools as a service. The main advantage of this service is the possibility to run different queries at the same time and combine their results in real time. Additionally, we also introduce twearch, a prototype to develop twitter mining algorithms as services in the cloud.Peer ReviewedPostprint (author’s final draft

Modelo en red de los contenidos mediáticos en la era de los dispositivos inteligentes

The changes in the information technologies of the last two decades, represented by the democratization of the Internet and the availability of portable and intelligent handhelds, is associated with new business, communication and communicative models. The high service capacities of today's smartphones have enabled a two-way dynamic processing of great amounts of content. The users gather in virtual communities and become both consumers and producers of information: prosumers. These contents come from or are allocated in the cloud or other platforms, and spread rapidly through the net. In this paper, the evolution of information fluxes is analyzed in three phases of the Information Society or Net Society, and the theory around user-generated content and media convergence is discussed. Each phase is represented in a cyclical model articulating data dealing, the new roles of users and the impact of technology. On the basis of these models, news discourses and content may be anticipated for various purposes. A flow model based on algorithms determined by the personal preferences of users is finally proposed.Los cambios en las tecnologías de la información en las últimas décadas, representados por la democratización de Internet y la disponibilidad de las computadoras portátiles y dispositivos inteligentes están asociados con nuevos negocios y modelos comunicativos. La capacidad de los actuales teléfonos inteligentes facilita el procesamiento bidireccional y dinámico de grandes cantidades de contenidos. Los usuarios se agrupan en comunidades virtuales y se convierten en consumidores y productores: prosumidores. Estos contenidos se encuentran en la nube, en los medios de comunicación (plataformas), y se propagan rápidamente a través de la red. En este artículo se analiza la evolución de estos flujos de mensajes en el marco de las teorías que rodean el contenido generado por el usuario y la convergencia de medios. Se analiza también la naturaleza cíclica de la secuencia de datos, la evolución de los usuarios y la dependencia tecnológica para desarrollar y distribuir nuevos productos discursivos Se propone finalmente un modelo de flujos basado en algoritmos determinados por las preferencias personales de los usuarios.Peer ReviewedPostprint (author's final draft

Dynamic configuration of partitioning in spark applications

Spark has become one of the main options for large-scale analytics running on top of shared-nothing clusters. This work aims to make a deep dive into the parallelism configuration and shed light on the behavior of parallel spark jobs. It is motivated by the fact that running a Spark application on all the available processors does not necessarily imply lower running time, while may entail waste of resources. We first propose analytical models for expressing the running time as a function of the number of machines employed. We then take another step, namely to present novel algorithms for configuring dynamic partitioning with a view to minimizing resource consumption without sacrificing running time beyond a user-defined limit. The problem we target is NP-hard. To tackle it, we propose a greedy approach after introducing the notions of dependency graphs and of the benefit from modifying the degree of partitioning at a stage; complementarily, we investigate a randomized approach. Our polynomial solutions are capable of judiciously use the resources that are potentially at user's disposal and strike interesting trade-offs between running time and resource consumption. Their efficiency is thoroughly investigated through experiments based on real execution data.Peer ReviewedPostprint (author's final draft